712CD 


73rd  MORSS  CD  Cover  Page 

UNCLASSIFIED  DISCLOSURE  FORM  CD  Presentation 


For  office  use  only  41205 


21-23  June  2005,  at  US  Military  Academy,  West  Point,  NY 

Please  complete  this  form  712CD  as  your  cover  page  to  your  electronic 
briefing  submission  to  the  MORSS  CD.  Do  not  fax  to  the  MORS  office. 


Author  Request  (To  be  completed  by  applicant)  -  The  following  author(s)  request  authority  to  disclose  the 
following  presentation  in  the  MORSS  Final  Report,  for  inclusion  on  the  MORSS  CD  and/or  posting  on  the 
MORS  web  site. 


Name  of  Principal  Author  and  all  other  author(s):  MAJ  Michael  Armstrong 
Principal  Author’s  Organization  and  address: 

Army  Test  and  Evaluation  Command  Phone:703-681-9295 

4501  Ford  Ave 

Alexandria,  VA  22302  Fax:  703-681-3498 

Email:michael.a.armstrong2@atec.army.mil 


Original  title  on  712  A/B:  Capability  Based  Evaluations 


Revised  title: 


Presented  in  (input  and  Bold  one):  (WG25,  CG _ ,  Special  Session _ ,  Poster,  Demo,  or  Tutorial): 


This  presentation  is  believed  to  be: 
UNCLASSIFIED  AND  APPROVED  FOR  PUBLIC  RELEASE 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

22  JUN  2005 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Capability  Based  Evaluations 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Army  Test  and  Evaluation  Command  4501  Ford  Ave  Alexandria,  VA 
22302 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM201946,  Military  Operations  Research  Society  Symposium  (73rd)  Held  in  West  Point,  NY  on 
21-23  June  2005. ,  The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

uu 

18.  NUMBER 
OF  PAGES 

13 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Capability  Based  Evaluations 


MAJ  Michael  Armstrong 
Army  Test  And  Evaluation  Command 


Army  Test  and  Evaluation  Command 


Why  Capabilities 
Based  Evaluation 


•  January  2002  SEC  DEF  cancelled 
Missile  Defense  Operational 
Requirements  Documents  (ORD) 

•  New  capabilities  were  taking  too  long 
to  get  in  the  hands  of  the  soldiers 

•  Requirement  were  holding  up  fielding 
of  needed  capabilities  in  the  field 

•  The  complete  T&E  process  was  too 
long 
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Impact  On  Change 

•  Patriot  Advanced  Capabilities  3  (PAC-3) 
Initial  Operational  Test  and  Evaluation 
(IOT&E)  was  in  its  execution  phase  of 
testing 

•  Results  of  the  IOT&E  had  to  be  briefed 
in  two  ways 

-  Requirements  Based 

-  Capabilities  Based 
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Solution  after  Change 

•  We  realized  two  things: 

-We  had  always  done  Capabilities  Based 
evaluations  but  it  was  done  at  the  decision 
maker  level 

•  Systems  were  bought  with  requirements 
not  met 

•  Systems  were  sometimes  not  bought  with 
requirements  met 

-  Met  and  Not  Met  is  what  “went  away” 

•  We  used  the  usual  system  measures  and  the 

answers  became  means  of  characterization 
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No  Requirements? 


•  PAC-3  ORD  cancelled 

•  Limited  Guidance 

•  No  examples  for  Capabilities  Based 
Evaluations 


System  ORDs  were  the  framework 
upon  which  evaluations  had  been  built 
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Where  is  the  Focus  Today? 


•  Capabilities  -  Main  Focus 

•  Limitations  -  Limited 

•  Test  Conduct  -  Limited 

•  Unknowns  -  Limited 
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Operational  Questions? 


Is  there  a  measuring  stick? 

What  is  the  minimum? 

Are  there  Thresholds? 

Are  there  Objectives? 

What  is  Spiral  Development? 
What  is  the  Block  approach? 

What  are  Acquisition  Increments? 
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Operational  Utility? 

•  Does  the  soldier  determine  operational 
utility  of  the  new  capabilities  ? 

•  Does  the  Program  Manager  determine 
the  users  needs? 

•  The  user  can  understand  operational 
utility  only  after  he  knows  in  detail  the 
new  capabilities  being  added  and  the 
limitations  associated  with  those 
capabilities. 
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Decision  Time 


•  What  constitutes  a  capability? 

•  Is  one  new  capability  worth  fielding? 

•  Is  adding  10%,20%,30%...  to  current 
capability  worth  fielding? 

•  Who  makes  the  decision  on  what  is 
good  enough? 
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What  Is  Good  Enough? 


•  How  do  you  measure  when  you  have 
no  measuring  device? 

•  What  will  the  measurement  mean 
without  a  standard? 

•  Will  the  new  capability  establish  a  new 
standard  or  will  each  modification 
become  its  own  standard? 
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Informing  The  User 

*  The  User  must  know  how: 

-  New  capabilities  impact  operations  (Operational 
Terms) 

-  Limitations  impact  operations  (Operational 
Terms) 

-  Unknowns  could  impact  operations 

•  Who  explains  to  the  user  how  the 
capabilities,  limitations  and  unknowns  were 
demonstrated  or  tested 

-  Operational  Testers/Evaluators? 

-  Program  Managers? 

-  Contractors? 


User  involvement  throughout  Test  and  Evaluation  Process 
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What  is  Important 

•  What  ever  capability  is  fielded,  we  must 
keep  in  mind  that  the  user  is  the  soldier  in 
the  field.  They  must  trust  and  know  their 
equipment’s  capability  in  order  to  be 
successful  on  today’s  modern  battlefield. 
We  the  test  &  evaluation  community  have  a 
responsibility  to  inform  the  soldier  in  the 
field  on  the  capabilities,  limitations,  and 
unknowns.  It  is  up  to  them  to  determine 
when  and  where  to  use  the  new  capability. 

Give  the  user  the  tools  to  make  a  informed  decision. 
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